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Abstract 

Several exact recovery criteria (ERG) ensuring that orthogonal matching pursuit (OMP) 
identifies the correct support of sparse signals have been developed in the last few years. 

These ERG rely on the restricted isometry property (RIP), the associated restricted isometry 
constant (RIG) and sometimes the restricted orthogonality constant (ROG). In this paper, 
three of the most recent ERG for OMP are examined. The contribution is to show that these 
ERG remain valid for a generalization of OMP, entitled simultaneous orthogonal matching 
pursuit (SOMP), that is capable to process several measurement vectors simultaneously and 
return a common support estimate for the underlying sparse vectors. The sharpness of the 
bounds is also briefly discussed in light of previous works focusing on OMP. 

1 Introduction 

Recovering a high dimensional sparse signal by acquiring it through a linear measurement pro¬ 
cess returning fewer observations than its dimension is a problem often encountered in the 
digital signal processing literature. The field of research associated with such problems is often 
known to researchers as compressed sensing or compressive sensing (CS) (ld |. 

We define the support of a vector ® G M” as supp(£c) := {j G [n] : Xj / 0} where [n] denotes 
the set {1, 2,... , n} and Xj denotes the jth entry of x. A vector is said to be s-sparse whenever 
its support exhibits a cardinality equal to or lower than s. 

1.1 Signal model 

In this paper, we focus on a framework involving 

1. K sparse signals G to be recovered (1 < < K), 

2. a common linear measurement process described by the matrix $ G 

3. K measurement vectors yi^ G M™' gathering the observations of each sparse signal when 

acquired through = ^Xk- 
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To simplify the signal model, we introduce Equation JT]) to summarize the K equations 
into a single one: 

Y = (1) 

where Y = [yi ,..., e and X = [xi ,..., xk) G Using this formulation, the 

support of X, denoted by supp(X), is equal to the joint support S := Ufcg[i^]supp(a;fc). 


When a model involves one measurement vector, it is referred to as a single measurement 
vector (SMV) model while models incorporating K > 1 measurement vectors are multiple mea¬ 
surement vector (MMV) models 14l |. 


The columns of $ are often referred to as the atoms. This terminology being typically asso¬ 
ciated with dictionaries, it is worth emphasizing that the problem of recovering a s-sparse vector 
X on the basis of the measurement vector y = ^x is equivalent to finding s columns (or atoms) 
of the (dictionary) matrix $ that fully express y when using the proper linear combination. 
The notion of atom will thus be used in the rest of this paper as it simplifies the mathematical 
discussions that follow. 


We now introduce additional notions that are used afterwards. For 0 < p < oo and x G M”, 
we define the norms ||a;||p := (X^j=i and ||ic||oo := \xj\. In this paper, every 

vector should be considered as a column vector. Also, for SC [n], the quantity xs denotes the 
vector formed by the entries of x indexed by S. Similarly, for a matrix $ G we dehne $5 

as the matrix formed by the columns of $ indexed within S. The Moore-Penrose pseudoinverse 
of any matrix $ is denoted by and its transpose is given by Finally, the inner product 
of two vectors x and y is written as (a;, y) and is equal to x^y. 


1.2 Simultaneous orthogonal matching pursuit 

Several algorithms exhibiting varying computational complexities have been investigated to ad¬ 
dress the problem above. For the SMV case, the greedy algorithm entitled orthogonal matching 
pursuit (OMP) is a classical choice because its complexity is lower than that of other 


algorithms such as .^i-minimization 


If the K sparse signals x^ possess similar supports, i.e., their joint support S := Ufcg[;^]supp(®fe) 
possesses a cardinality that is comparable to those of the individual supports sup p(at-), then it 
is interesting to perform a joint and common estimation of their supports 1Q|, [lOj]. The simulta¬ 
neous orthogonal matching pursuit (SOMP) algorithm [^, which is described in Algorithm [H 
is an extension of OMP to the MMV case and performs a joint support recovery. 

As shown in Algorithm [U at each iteration t, SOMP adds to the estimated support the 
index jt of the atom maximizing the metric ||||i = \ {4>j,r^l'^)\ (steps 4 and 

5) where denotes the feth column of the residual matrix Each measurement vector yj^ 
is then projected onto the orthogonal complement of span($ 5 j^,^), denoted by span($ 5 j^,^)-*-, 
during steps 6 and 7. The algorithm terminates when the prescribed number of iterations s has 
been reached. It is worth noticing that an atom cannot be picked twice as, once chosen, the 
projection onto span($ 5 ^_^J-*- ensures that (0, = 0 if 0 G S'*. 


1.3 Defiuitious 

We define the concepts needed to state the results of Section [21 First of all, the matrix $ is said 
to satisfy the restricted isometry property (RIP) 0] of order s with restricted isometry constant 
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Algorithm 1: 

Simultaneous orthogonal matching pursuit (SOMP) 


Require: Y G $ G s > 1 

1 

Initialization: ^ Y and S'o ^ 0 

2 

t ^ 0 

3 

while t < s do 

4 

Determine the atom of to be included in the support: 
jt ^ argmax^g[„](||(Rfo)T</,^.||i) 

5 

Update the support : S't+i ^ S'* U {jt} 

6 

Projection of each measurement vector onto span($Sj^j): 

7 

Projection of each measurement vector onto span(# 5 j^j)-*- : 
jj(t+i) ^ Y — y (*+i) 

8 

t i — t -j- 1 

9 

end while 

10 

return Ss {Support at last step} 


(RIC) 6s (of order s) whenever 

(1 - (5s)||'u||| < ll^ulli < (1 + 5s)||u||i (2) 

holds for all s-sparse vectors u. Thus, the RIP ensures that the linear operator $ maintains the 
^ 2 -norm of s-sparse signals up to a certain extent that is quantified by means of the RIC Sg- Fur¬ 
thermore, if u is supported onto S, the quantity ||^w||| is equal to = Ug{^'g^s)us- 

The RIP therefore ensures that 1 — < Amin(^ 5 ^s) < Amax(^ 5 ^s) < 1 + for all the 

supports S of cardinality equal to or lower than s where Amin and Amax denote the minimal and 
maximal eigenvalues, respectively. Also, it is easy to show that 6s < (5s+i. 

The (a, a')-restricted orthogonality constant (ROC) Q] is defined as the smallest real number 
for which 

|($c,$c')| < 6 '„,q:'||c|| 2 ||c '||2 (3) 

holds for every c, c' G M™' exhibiting disjoint supports of cardinality a and a', respectively. 
Thus, the ROC quantifies how vectors with disjoint supports stay approximately orthogonal 
after projection by 

The ROC and the RIP are linked by the inequality [3, Lemma 2.1] 9a^a' ^ ^a+a' which 
indicates that the RIC can play a role similar to that of the ROC, albeit in a less sharp 
manner. Another similar inequality has been obtained in [ 2 ^ . Section 2.3] and is given by 
Ol a.' < \/a'Ka' — l)(5r>' whenever a' > 2 . Another upper bound of has been recently ob¬ 
tained in |28l . Lemma 11.3] where the so-called 2-coherence of the dictionnary matrix, denoted 
by r'd', is used. They have shown (iil . Lemma 11.2] that Va' < so that the inequality 

^i,a' < t'a' is sharper than Oi^a' < The developments presented hereafter use the RIC- 

based inequality so that only the RIC intervenes in the final results. However, expressing our 
results using Ua/ instead of di+a' is straightforward. 

Finally, it is worth defining the ^oo-induced norm for matrices as ||$||oo-).oo := sup||^i| ,_i ||$(?!)||oo 
(where $ G that can be computed as ||$||oo->.oo = Lemma 

A.5]. This quantity is interesting as it allows to write, for A C [n], maxjeyi(||(i?^d_ 
which is reminiscent of the decision metric of SOMP. Some authors choose to 
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write the ^oo-induced norm of $ as ||^||cxd but, to avoid confusions, we prefer to emphasize the 
distinction between the ^oo-norms for vectors and matrices as both coexist in Section SI 


2 Contribution and related work 


The main contribution of this paper is to extend a recent exact recovery criterion (ERC) for 
OMP to its MMV counterpart, i.e., SOMP. An ERC is a sufficient condition to ensure that the 
algorithm commits no mistake. The cornerstone of the results presented in this paper is given 
by Lemma [TJ 

Lemma 1 (A RIP and ROC-based lower bound on the maximal residual projection). Let 
X S possess the support S. Let $ £ j^mxn JiJC (5|g| < 1 and the (1, \S\)-ROC 

< 1. Furthermore, = ^St^St denotes the orthogonal projector onto span($ 5 j where 
St C S, i.e., only correct atoms have been included to the estimated support before iteration t. 
Let Rdl be equal to (I - pdl)Y = {I - Then, 


where S is the relative eomplement of S with respeet to [n]. 

Lemma [1] establishes a lower bound on the ratio of the SOMP metric obtained for the 
correct atoms to that obtained for the incorrect ones. In that sense, and as it will be clarified in 
Theorem [H it straightforwardly provides an ERC guaranteeing that SOMP commits no error 
when picking atoms. We now propose a corollary of Lemma [1] that only relies on the RIC. 

Lemma 2 (RIP lower bounds on the maximal residual projection). Let X £ possess the 

support S. Let $ £ admit the RIC 5\g\ < 1. Furthermore, P^'i = denotes the 

orthogonal projector onto span($ 5 j) where St C S, i.e., only correet atoms have been included 
to the estimated support before iteration t. Let pd'l be equal to (/ - pd))Y = {I - 
Then, both inequalities below hold 






> 
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where S is the relative complement of S with respect to [re]. 

Compared to former works that directly derived the ERC [3, 0, 23-25], we believe that 
Lemma [T] and Lemma [2] are interesting as they quantify the robustness of the decisions made 
at each iteration of SOMP in the noiseless case. Such quantities can then be used to produce 
theoretical analyses of greedy algorithms in a noisy setting (se e |il.Q.I l^L Other similar works 


analysing OMP in a noisy environment include 21[ and 


The analysis presented in 
actually uses a result fundamentally identical to Lemma [T] for A = 1 to conduct a theoretical 
analysis of OMP inspired from [ij] . We now state the three ERC deriving from Lemma [1] and 
Lemma [2j 
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Theorem 1 (Several RIP and ROC-based ERC for SOMP). Let X G possess the support 

S. Let $ G admit the RIC (5|5| < 1 and the (1, \S\)-ROC < 1. Then, SOMP commits 

no error and identifies the full support of X at the end of iteration |5| — 1 whenever at least 
one of the three conditions below hold: 




> 1 


(ERCl) 




(ERC2) 

(ERC3) 


As demonstrated in Section 1 4.3 1 Theorem [T] is a straightforward consequence of Lemma [T] 
and Lemma [2j The authors of [24i | and 0 independently obtained (jEEClD for OMP. To the 
best of the authors’ knowledge, the second ERC was first obtained simultaneously in [l^ and 
251 while (IERC3h was initially published in [^, both ERC being derived for OMP. 


Regarding older works, it is also worth pointing out that the ERC (5|5|_|_i < l/((l+\/2)-\/[5[), 
first obtained in 17], Theorem 5.2] for OMP, has been shown to remain valid for SOMP in [111 . 
Corollary Ij. Thereby, the authors of 11 j also proved that the older ERC 5|5|+i < 1/(3 y^| 5|), 
initially derived in Theorem 3.1] for OMP, remains correct for SOMP as 5|s|+i < 1/(3 y'^[^) 


is implied by 5|5|+i < 1/((1 + \/2)\/[^)- Very recently, (IERC2P was extended to SOMP in [271, 
Remark 1]. However, the extension to SOMP of both (lERCip and (IERC3P is a novel result. 
In [I^, the author has derived the ERC (5|5|+i < 1/^\S\ + 1, which is sharper than ()ERC2p . 
Combining the ideas developed in [l^ and our paper could possibly extend this ERC to SOMP. 


Finally, we would like to point out that, if any of the considered ERC holds, running K 
independent executions of OMP instead of a single instance of SOMP would enable one to 
retrieve the individual supports supp(®fc) {1 <k < K) and, by extension, the joint support S. 
While it may seem to undermine the interest of this work, the following observations convince 
otherwise: 

1. If one of the considered ERC guarantees that each one of the K instances of OMP returns 
the correct support of each sparse vector Xk, then SOMP is also guaranteed to return 
the correct joint support so that there is no penalty switching from OMP to SOMP, 
except maybe that SOMP returns a joint support instead of possibly smaller (yet correct) 
supports for each xj.. 

2. Lemma [Hand Lemma [2] should be thought of as the central results of this paper as they 
quantify the robustness of the support recovery in the noiseless case, the resulting ERC 
being merely direct consequences of the aforementioned lemmas. As mentioned previously, 
these lemmas can be used to produce theoretical analyses of SOMP for noisy scenarios 
while it is not the case for the ERC. 


3 Sharpness of the bounds 

In [^, it is shown that (|ERC1I) is sharp for OMP in the sense that it is possible to construct 
a measurement matrix $bad satisfying (1 — 5|5 |)/(0 i,|5|-\/[5T) = 1 for which there exists a |S'|- 
sparse signal iCbad that OMP fails to recover. The sharpness property is immediately extended 
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to SOMP by noticing that if OMP fails to recover a^bad on the basis of the measurement vector 
Vb^d = ^bad^bad, then SOMP also fails with Y bad = ^bad^bad where Xbad = (®bad, ■ ■ ■, ®bad) 
as both algorithms make the same decisions in this case. 


Regarding (IERC2P and (IERC3I) . it has been shown in [1^ that there exists a signal aibad 
of support S and a matrix $bad satisfying (5|5|_|_i = 1/■\/\S\ + 1 for which OMP fails to recover 
the support of a^bad on the basis of ybad = ^bad^bad- Note that earlier works (see (l9| and 
(iij ll proved that the statement above holds with $bad satisfying 5|5|+i = Using an 

approach identical to that of the previous paragraph, one shows that this statement remains 
true for SOMP with Ubad = ^bad^bad and Xbad = (®bad, • • • ,®bad)- R shows that (IERC2P 
is near-optimal as, for |5| —>■ oo, it boils down to the condition (5|5|_|_i < l/y^|5| -|- 1. It can be 
shown that (|ER.C3h is also near-optimal but the discussion is more involved as 5|5| intervenes 
instead of (^|s|+i. In [^, Section 3], it is shown that (5|5|+i < 1/(|<S'| -|- 3 — \/2) implies (IERC3p . 
therefore indicating that ()EE,C3h is also at least near-optimal. 


4 Proofs 

4.1 Proof of Lemma [T] 

The proof presented in this section is analog to what has been proposed in [^. 241. the only 

difference being the additional quantities needed to deal with the MMV model. 


The proof is decomposed in three steps: 


1. Derive an upper bound on 


expressed as ||:z''“q |2 


where z^^'> is to be 

specified in the detailed development. 

2. Derive a lower bound on ||oo_^oo expressed as (l/-yi(^)(l — (5|5|)||z(*^ II 2 where 

is identical for steps 1) and 2). 

3. Compute the ratio of the lower bound to the upper bound and observe that the desired 
result is obtained due to the cancellation of the quantity ||z*^*)|| 2 . 

Let us first tackle the quantity ||oo->.cx) = I ^i)l) define 

j*{t) := aiguia.-^-^-g{Yjk=i\(f'f ^^j)\)- Then, if cf := sign((r[*\ we have 


K 




|oo—)-oo — ni8X 


Eit 


vs 

K 

M/Ji) 






k=l 

(e 




Since St C S, = {I — belongs to span(4*5) and can thus be expressed as a linear 

combination of the atoms whose indexes belong to S by means of where S 

contains the coefficients of the linear combination of interest. It is also worth defining the 
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extension of to M” by ensuring that supp(a^*^) C S and Another 

relation of interest is 4>j*(^t) = where denotes the j*{t)th vector of the canonical 

basis of M”. Hence, using consecutively the equations of this paragraph and the definition of 
the ROC yields 




K 






k=l 

K 

E 

k=l 


Cfc “fc 


where ||ej*(t )||2 is equal to 1. It is worth explicitly pointing out that the ROC definition is 
applicable in that case because the supports of and Ylk=i ^^k^^k disjoint as j*{t) £ S 
and supp(d^*^) C 5 for 1 < /c < AT. 


The first step of the proof is now completed and the last problem to be dealt with is deriving a 
lower bound for ||oo-).oo- For any £ {—1; 1}, we have \{r^l\4>j)\ = 4> j)\ = 

0j)|. In particular, it remains true for the choice Thus, by using the 

equation above and the triangle inequality, one obtains 



where ^"s^^k=i^^k "^^k) ^ Also, we have previously obtained The lower 

bound on ||oo->-cxd is thus finally obtained by successively using the two previous relations 

and the inequality 1—d| 5 | < Amin(^ 5 ^s) resulting from the RIP (see Section fL3]l in the following 
manner: 



where || Ylk=i^^k^^k\\‘i ~ II ^ lb- The final result is now established by expressing 

the ratio of the lower bound on H^^i^^^bloo-^-oo to the upper bound on □ 
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4.2 Proof of Lemma [2] 

The proof consists in finding lower bounds on the ratio (1 — <5|5|)/(0i,|5|\/[5T) intervening in 
LemmaJH For the first bound, it is sufficient to use the inequalities (5|5| < 5|5|+i and < 
<^|S|+i Id) Lemma 2.1], for the numerator and the denominator respectively. The second bound 
is obtained by using the inequality < y/|5|/(|5| — 1)(5|5| on the denominator for |5| > 2 
0, Section 2.3]. ’ □ 

4.3 Proof of Theorem [1] 

Let us first address the proof of (lERCip . At iteration 0, we have = T and Lemma [T] shows 
that a sufficient condition for SOMP to pick a correct atom is (1 — 5|5 |)/(0 i |5 | y^]^) > 1 as it 
means that the highest metric is necessarily obtained for one of the correct atoms. Thus, at 
iteration 1, the condition 5i C 5 is verified and Lemma [T] shows, once again, that a correct 
decision will be made. By repeatedly applying the same train of thought, one proves the theorem 
by induction. The remaining ERC are obtained in an identical manner by using the two bounds 
provided by Lemma [2] instead of that of Lemma [TJ □ 
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